Assessing the Learning and Transfer of Data Collection Inquiry Skills Using Educational Data Mining on Students’ Log Files
نویسندگان
چکیده
In this paper we explored whether engaging in two inquiry skills associated with data collection, designing controlled experiments and testing stated hypotheses, within microworlds for one physical science domain (density) impacted the acquisition of inquiry skills in another domain (phase change). To do so, we leveraged educational data mining techniques to both assess and estimate students’ inquiry skills across domains. Analyses revealed that honing these skills in density activities provided benefits in terms of transfer and skill acquisition. More specifically, students who practiced in density activities first were more likely to show mastery of the designing controlled experiments skill than those who had no prior practice. These same students were also more likely to test their stated hypotheses during their first data collection in phase change. Thus, practice in one domain can positively impact acquisition and transfer of skill in a second domain, suggesting that inquiry skills also have a degree of domain generality. Introduction Science educators agree that cultivating inquiry skills is critical for students to become scientifically literate (National Research Council, 1996, 2000, 2011; Kuhn, 2005a). However, typical standardized science tests do not adequately reflect or assess complex inquiry process skills (Quellmalz, Timms & Schneider, 2009). Performance assessments of inquiry, instead, have been argued to be better-suited for this purpose (cf. Black, 1999; Pellegrino, 2001). Devising scalable and reliable performance assessments, though, is difficult for two reasons. First, it is difficult to separate inquiry skills from content understanding (Mislevy, Steinberg & Almond, 2002; Mislevy, et al., 2003). Second, inquiry processes are mutli-faceted, and there is no one single “right or wrong” way to engage in science inquiry (Shute, Glaser & Raghavan, 1989; Glaser, Schauble, Raghavan, & Zeitz, 1992). Given inquiry’s importance, proper techniques for measuring inquiry are needed. It is also important to better understand inquiry learning so that we can foster transfer of such skills to novel tasks (Kirschner, Sweller & Clark, 2006; Hmelo-Silver, Duncan & Chinn, 2007). Regarding transfer, it has been suggested that inquiry skills are tightly tied to the domain in which they are learned (van Joolingen, de Jong & Dimitrakopoulout, 2007), but some evidence exists that long-term, repeated practice of inquiry (Kuhn, Schauble & Garcia-Mila, 1992; Dean Jr. & Kuhn, 2006; Kuhn & Pease, 2008), and scaffolding or teaching these skills explicitly (Klahr & Nigam, 2004) can lead to successful acquisition and transfer to novel tasks. In the present paper, we address two goals. First, we describe our approach for developing reliable, scalable performance measures of inquiry. Second we leverage those assessment techniques to examine how inquiry skills transfer between two physical science domains. We focus on on two inquiry skills, designing controlled experiments and testing stated hypotheses. Designing controlled experiments entails selecting experiments to yield data that supports determining the effects of manipulable variables on outcomes. Testing stated hypotheses refers to generating data with the intention to support or refute a specific hypothesis. These skills are measured as students conduct inquiry within microworlds for two domains, phase change and density, developed within the Science Assistments system (Gobert et al., 2007; Gobert et al., 2009). In our approach, we leverage techniques from Educational Data Mining (cf. Baker & Yacef, 2009; Romero & Ventura, 2010) to assess and track inquiry skills across several activities within each domain. To assess these skills, we use validated detectors (models) of students’ inquiry behaviors that were constructed based on student log files (Sao Pedro et al., 2010, in press). We then produce estimates of student proficiency for each skill by aggregating all assessments into a Bayseian Knowledge Tracing model (Corbett & Anderson, 1995). This approach is rigorous because an EDM affords the ability to estimate how well the models assess and track skill. Furthermore, these can be done in real time. Thus, we argue, the approach is scalable, and could provide a possible model for inquiry assessment. These techniques were also leveraged to measure whether the two skills of interest, designing controlled experiments and testing stated hypotheses, transfer across two physical science domains. These two domains are Density and Phase Change. More specifically, we analyzed whether students who practiced in density activities first had a greater likelihood of demonstrating the skills or reaching mastery than students with no prior practice. With our assessment and skill tracking models, we can assess this transfer in a finer-grained way than other prior studies of inquiry (e.g. Kuhn & Pease, 2008). The remainder of this paper is organized as follows. First, we describe the two skills of interest in more detail, and present related work on assessing data collection skills. We then present a high-level view of our approach for assessing and estimating proficiency at these data collection skills using our educational data mining techniques. Next, we present our results on the transfer of these skills between domains which leveraged our assessment and estimation techniques. Finally, we present a discussion and conclusions of our paper.
منابع مشابه
Identifying Transfer of Inquiry Skills across Physical Science Simulations using Educational Data Mining
Students conducted inquiry using simulations within a rich learning environment for 4 science topics. By applying educational data mining to students’ log data, assessment metrics were generated for two key inqury skills, testing stated hypotheses and designing controlled experiments. Three models were then developed to analyze the transfer of these inquiry skills between science topics. Model ...
متن کاملA Clustering Methodology of Web Log Data for Learning Management Systems
Learning Management Systems (LMS) collect large amounts of data. Data mining techniques can be applied to analyse their web data log files. The instructors may use this data for assessing and measuring their courses. In this respect, we have proposed a methodology for analysing LMS courses and students’ activity. This methodology uses a Markov CLustering (MCL) algorithm for clustering the stude...
متن کاملThe Science Assistments Project: Intelligent tutoring for scientific inquiry skills
Our learning environment Science Assistments (www.scienceassistments.org; NSF-DRL# 0733286; NSFDGE# 0742503; NSF-DRL# 1008649; U.S. Dept of Ed.# R305A090170) scaffolds middle school students’ scientific processes, namely, hypothesizing, design of experiments, data interpretation, warranting claims with evidence, and communicating findings for Physical Science, Life Science, and Earth Science. W...
متن کاملEducational Data Mining and Problem-Based Learning
This paper considers the use of log data provided by learning management systems when studying whether students obey the problem-based learning (PBL) method. Log analysis turns out to be a valuable tool in measuring the use of the learning material of interest. It gives reliable figures concerning not only the number of use sessions but also the interlocking of various course activities. The lo...
متن کاملComparison of the Effect of Case Scenario Teaching Methods and Concept Map on Learning Cardiopulmonary Resuscitation Skills of Children in Nursing Students
Background and Objectives: Nurses must be sufficiently skilled in performing cardiopulmonary resuscitation in hospitalized patients. The aim of this study was to compare the effect of case scenario teaching methods and concept map on learning cardiopulmonary resuscitation skills among nursing students. Materials and Methods: In this quasi-experimental study consisting two-groups, before-after ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2012